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Qvrviam 

During 1982> in conjunction with NA8A/CSFC Softworo 
Enginooring Lohorotory (SEL)> rosoorch was conducted in 4 
areas: Software Development Predictors* Error Analysis* 

Reliability Models and Software Metric Analysis. Summaries 
of the projects follow below. 

1. agffadAH Development Erf.flitt.9Xl 

A study is being done on the use of dynamic charac- 
teristics as predictors for software development. It is 
hoped that by examining a set of readily available charac- 
teristics* the project manager may be able to determine such 
things as when a project is in trouble and evaluate the 
quality of the product as it is being designed. 

Project DEB was selected as the control for the project 
since it was considered fairly successful and is well docu- 
mented. Information found in the history files and resource 
summary files was initially utilized. These files were 
chosen because the information they contain is readily 
accessible to the manager (ie. number of lines of code* 
manpower* computer time* etc ). Several profiles of project 
DEB were then made using this information. Project OEA's 
profiles were then compared with these results. This pro- 
ject was chosen because it was vory similar to DEB but was 
considered less successful. 
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Th« hivtory flit was first •laminad to too if any 
growth pattorn oxittod for tho linot of coda. Tho initial 
look at OEA and DEB lookod hopoful but furthor invott igation 
of othor projocto thowod no ditcorniblo pattorn. Othor 
oxaminations of this filo yioldod similar rosults. 

Mhon a comparison of the information in tho history and 
rotourco summary filos was mado somo difforoncos did appear. 
Initial plots used accumulative totals versus different time 
factors. These plots did demonstrate visible differences 
between the two projects. Further investigation using 
weekly totals instead of accumulative totals showed an even 
larger difference between the projects. 

Project DEA had a higher frei^uency of changes at 
the beginning of the project/ while at the same time/ the 
number of hours of manpower reported for the interval was 
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From this tnaainatlon th* following gonorol goals 
and hgpothasit hava boon ganorated: 

A) Tha manpowar usaga in tha SEL anvironmant is a discarni' 
bla pattarn and nay ba utad as a pradictor. 

1) Tha idaal staffing fo;* a succassful projact is a two 
hump curva with tha sacond hump baginning roughly 2/3 
into tha projact. 

2) Tha two humps mantionad in hypothasis 1 should paak 
at approximataly tha sama height. 

3) Tha maximum paak haight of tha first hump is propor- 
tional to tha final siza of tha projact. This also hold 
for the sacond hump based on hypothesis two. 

4) Tha location of tha two peaks is constant with rela- 
tion to tha amount of manpower utilizad. 

5) Tha amount of manpower expanded between tha two 
peaks is constant. 

6) Projects deemed less succassful by subjective 
analysis hava sharp changes in the amount of manpower 
spent par change. 

B) Tha pattern of changes in relation to manpowar> computer 
runsi lines of codai etc. may be used as a predictor in the 
SEL environment. 

1) Tha amount of manpower to make a change should 
increase toward tha and of a project and be stable at 
tha beginning. 

3) The manpowar par change should ba lower in tha 
beginning of tha project. Sea also goal D. 

3) Projects daamad lass succassful by subjective 
analysis hava sharp changes in the amount of manpower 
spent per change. 

4) Tha ratio of changes to computer run should decrease 
as the project evolves 

5) The amount of computer time spent on detecting and 
correcting a given change will remain constant 

C> The number of computer runs is closely related to the 
development of a project and may be used to judge project 
development. 
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1) Tht nuAbtr of computor runt rtmaint constant during 
tho initial huap of tho staffing curvo. Tho nusibor of 
computor runs will drop during ths socond hump of tho 
staffing curvo. 

2) Tho ratio of changes to computor runs should 
docroaso as tho project evolves. 

D) A close eiamination of the types of changes and the pat- 
tern they make over time should be a good indication of the 
success of a given project. 

1) Time consuming charges that occur late in the pro- 
ject more often appear in modified code. 

2) Unit testing is not as entensive on modules with 
modified codo. Undetected errors may cause major prob- 
lems latter in development. 

3) The types of changes vary across the development of 
a pro ject. 

4) The number of changes per houT- of manpower is 
related to the type of changes being done. 

5) The types of change that require more time to 
correct occur during the second staffing hump. 


Several projects will now examined to test the validity 
of these finds. The change report forms will also be exam- 
ined to see if the information in them yields any useful 
predictors. 


To conclude* the study has completed its initial 
analysis of the two projects. It appears there are some 
significant factors that could be useful as predictors. 
Further analysis may yield some information that would be 
useful to a project manager. 
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2- EcrfiL ftniUlli 

A) . Publication of oxiating rosulta — Throo papara art 
balng prtparad from oarlit r work on trror analyaia conducted 
by the SEL laboratory. One ia on the data collection 
methodology and the validation of the accuracy of the data# 
the aecond one ia on the analysis of the SEL projects 
directly and the third one is a compa^'ison of the SEL pro- 
jects with projects of the Naval Research Laboratory. These 
papers are currently being submitted for publication and 
will be published as University of Maryland Icchnical 
Reports in the interim. 

B) . A study on software errors and complexity — The dis- 
tribution and relationships derived from the change data 
collected during the development of the medium scale satel- 
lite project shows that meaningful results can be obtained 
which allow insight into software traits and the environment 
in which it is developed The project studied in this case 
was QMAS. Modified and new modules were shown to behave 
similarly. An abstract c lass if ication scheme for errors 
which allows a better understand ing of the overall traits of 
a software project was also provided. Finally/ various size 
and complexity metrics are examined with respect to errors 
detected within the software yielding some interesting 
results. A University of Maryland Technical Report describ- 
ing these results was published CBasSe]. This paper has been 
submitted for publication. 
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C). A furthtr •R«mln«tion of tho orror charactor ittict of 
tho OE.A and OE^ projacti it currantig baing undartakan. 
Thit arror analgtit it baing conductad uting tba tachniquat 
davalopad and documantad in CWaiSl] and CPar623. Tha focal 
point of thit rataarch affort it to charactariza arrort in 
tha NA8A/0SFC toftwara davalopment anvironmant. 

A prtlifflinarg raviaw of a tampla of tha Changa Raport 
Forfflt from both DE_A and DE_B hat baan conductad. Tha tarn- 
pla includad only thota CRF't for which an arror changa wat 
raportad. Tha pw^pota of thit reviaw wat to ^gat a flavor' 
for tha data collactad and to preliminarily attctt tha con- 
tittancy of that data with tha ratultt found to data by SEL 
partonnal. 

Tha tampla includad 98 CRF's from DE_A and 90 CRF's 
from DE_B. Of tha 98 CRF't from 0E_A. 63 (64 3%) of tha 
arrort wara clattifiad at an 'arror in tha datign or imple- 
mantation of a tingla componant. ' Of tha 90 CRF't from 
DE_B. 16 arrort wara raportad at 'clarical arrort. ' Of tha 
ramaining 74 DE_B arrort (non-clerical arrort). 61 (84. 2X) 
of tha arrort wara alto clattifiad at 'arrort in tha datign 
or imp lamantation of a tingla componant. ' 

Although tha parcantaga clattifiad at 'arrort in a tin 
gla componant' for DE_B wat higher than tha othar ttudiat. 
thata praliminary ratultt appaar to follow tha ratultt of 
praviout analytat CUai811. At in that praviout work, tha 
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diftrlbutlon of orrort in ethtr catogorito dots not noatly 
fit a pottorn. In facti thara ara too faw avanta in tha 
othar catagoriat to draw any initial concluaiona. It will 
ba intaraating to aiplora tha raaaon(a) DEJB aapariancad a 
aubatantial ly largar nuMbar of 'clarical arrora. ' 

1hara ara markad diffarancaa in tna ramaining DE_A and 
DE_B arror raporta. Thia may ba attributabla to tha 
raportad diffarancaa in tha two projacta. It ia not poaai- 
bla at thia tima to conjactura on mora tangibla cauaaa for 
tha diffarancaa. Tha full aat of arror changa raporta will 
hava to ba axaminadi for both projacta. 

It ia worth noting hara that for DE_A« 31 of 98 arror 
raporta (31.6%) axaminad were riaaaifiad aa being an 'error 
in tha daaign or implamantation of more than one component. ' 
Baaed on previoua raaulta cited above, thia ia an unuaually 
high parcantaga. Only 4 componanta (4 1%) had arrora 
raportad that ware not in tha design or im;:lamantation of 
componant(a) catagoriaa. 

Aa part of tha preliminary work toward tha above goal, 
tha related literature relaaaad by SEL waa reviewed. A con- 
cluaion reached waa that tha dafinitiona of aavaral critical 
tarma ware not nacaaaarily conaiatant. and often timaa tha 
technical raporta make too great an aaaumption about tha 
uniformity of uaa of aoftwara engineering tarma. 
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'Inttrf«c«' provldtt « good oxAinplo of an ill-dtfintd 
vat oft utad tarm. Oting tha dafinition from CUaiSl] (t>a 
aama dafinition it utad in CBatSOb] and C01o793) it it argu 
abla that intarfaca arrort can ba capturad fiva wayt from 
tha CRF: 

-an arror involving nora than ona componant; 

-an arror involving a common routinai 

-from taxtual coaimantt in tha CRF (ag: a CRF for which 
tha arror wat antarad at having affactad ona componant 
but tha taxt indicatad that tha arror wat in a tubrou- 
tina call ttatamant)< 

-an arror raportad at having baan locatad in ona com- 
ponant but tha changa raquirad to rapair tha arror 
affactad mora than ona componant> and 

-a changa that cautad an arror btcauta aithar tha 
changa invalidatad an assumption mada alsawhara in tha 
softwara or an attumption mada about tha ratt of tha 
toftwara in tha datign of tha changa wat incorract 
(contingant on abilitg to capture supporting taxt and 
ability to distinguish from arronaous assumptions mada 
about a single componant). 


An effort is currently underway to develop a mora res- 
trictive sat of definitions for software *,near ing terms* 
specifically those that apply to error analysis. Tha basis 
of this effort it tha sat of definitions published in 
CBasSOl and C01o793 and will ba modifiadi as nacassary# in 
consultation with those parsons associated with SEL in the 
past and prasanti whose work is or was related to tha arror 
analysis effort. 
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a. Bi.iijLkiu&u. Oflifiii. 

A «tudy it bting performed in the «r«« of rollability 
Modolt. Thif rttoarch includtt tht flold of program tooting 
locouto tho validity of torn# roliability modtlt dopondi on 
tbo anoworo to tom# unanoworod quottiont about tooting. 

Tho ovontual goal of thio roooarch io to undorotand hou 
and whon to uoo roliability modolo. Uo aro inv/ostlgating 
tho uoo of functional tooting bocauoo somo roliability 
modolo mako aooumptiono about the way program tooting io 
accompllohod CMuoal. It io not known if functional tooting 
oatiofioo the random tooting aosumptiono mado by tho relia^' 
bility modelo. Tho validity of roliability modolo that uoe 
data gonoratod by functional testing io uncertain until thio 
question io anoworod. 

W,e are using structural coverage metrics to gain 
further insight into the effects of functional testing. A 
structural coverage metric is a measure of how muen of a 
program was executed for given input data. Studying the 
coverage metric miy allow uo to develop other moaouroo of 
reliability. 

An additional bonus of this research io that it allows 
uo to compare functional testing and structural testing. It 
io not known how these two methods of testing are related. 
Tho results of thio investigation may answer that question. 
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Sinct January background matorlal hat boon ttudiad with 
ragard to p'oliability modolti and functional and structural 
tasting CMuallarl. A FOf?TRAN preproctssor has baan writtan 
to calculata tha structural covcraga matrics of 08FC FORTRAN 
sourca coda. 

lha praprocatsor calculatas tha simplast matric< tha 
parcant of axacutabla coda that is asacutad. Thara ara 
savaral ways to aiaasura covarago CAcarbachl. Ona mathod 
usas Intarpratation of tha sourca coda. Tha intarpratav 
racords which statamants ara exacutad. At tha and of 
intarpratat ion< it writas a list of axacutad statamants 

Tha sacond mathcd usas '$wi.t;chas"< small sactions of 
coda that ara insartad into tha sourca program taxt wharavar 
tha flow of control divargas or convargas. Tha switch has 2 
valuas: 0 if it was not axacutad. 1 if it was axacutad. Tha 

valua of tha switchas is output aftar axacutior. 

All axampla: 

INTEGER SWITCH ( N ) 

FOR I - 1, N 

SWITCH (I) ■ 0 

READ ( J )i 

If ( avan ( J > ) 

THEN 

SWITCH ( 1 / • 1; 


ELSE 


SWITCH ( 2 ) « 1; 
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ENDIF 

FOR I ■ 1, N 

WRITE ( SWITCH ( I )>; 

END 

When thit progr«« it •x«cut«d< on« of tho two bronchof 
of th« if «tott«ont will bt txteutod. tKOoiining tho 

voluts cf tht orrau SWITCH^ wo can dotormlno what codt was 
oxocutod. Bg analyilng tho coda and counting ttatofflont«< 
tht nuoibtr of ttatomonto oxocutad can bt dtttrmintd. In 
practictf tht amount of data gtnsrattd will bt largo. 
Softwart toclt art nttdtd to htip anaigzt th# data. 

Iht switchot can bt instrtad bg a prtproctttor (btfora 
compilation) or bg a compiler (during compilation). Tht 
switchot mag bt in-lint codt (as in th# txamplt) or a call 
to a twitch tubroutint that rtcordt th« flow of control 

Thit latter approach wat taken and a prtproctttor wat 
dtvtloptd that runt on VAX/Unix at UUCP. Tht prtproctttor 
taktt a copg of the input tourct codt. and modifitt it 
Thit modified copg will bt returned to the tourct computer 
(at (9SFC) where it will bt compiled and executed Tht exe- 
cution products th« desired coverage data. The coverage 
data will be returned to the Univerti^g for analgtit 

Hang things remain to be dune before we reach our goal 
of understanding how and when to use **eliabilitg models 
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Th« liMitdiatt go«l it to trg to ontwor tho functional to«t** 
ing / rollobllltg Modol quottion. Tho projoct RAOHAS hot 
boon choton ot on oiportmontol tgttoo CC8C3. Tho proprocot* 
tor fflutt bo utod to modifM tho RA0MA8 tourco codo. (Tho 
RA0MA8 projoct ond itt f unctionol lg*gonorotod occoptonco 
tottt hovo boon «odo ovoiloblo for tho covorogo oiporimont. > 
Tho modifiod RADMAS codo mutt be oiocutod ot GSFC uting t'.io 
funct ionol 1 y-gonorotod occoptonco toot*. 

Ihit oiporimont thould ontuor thofo quottiont obout 

functionol totting ond roliobility modolt: 

*Uhot it tho porcont covorogo of furctionol totting? 

-Ooot functional totting meet the rondomnett roquiro* 
montt 

of tho MTTF modolt? If not. con it bo modo to? 

*0o tho ttructurol motnct thow jny utoful pottornt in 

tho woy that functional totting tottt programt"^ How 

doot tho covorogo tot grow"^ At whot roto doot tho cov- 
orogo tot grow? 

-How indopondi»nt oro individuol tottt from o covorogo 
point of viow? 

Tho rotultt of thit oiporimont will raito furthor quot* 
tiont about functional totting ond roliobility modolt Thit 
will roquiro moro oipor imontotion. If thoto quottiont oro 
antworcd. thoro it moro work to do concorning how ond whon 
to uto roliobility modolt. 
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Tht «ttr«c%ion of tht abUity to prtdict tho offort in 
dovtloping or oiplain tha quality of aoftiMro hat lad to tha 
propotal of tavaral thaoriat and matrict CHal77« McC76» Oaf. 
Cha78. Cur793. In tha Softwara Cnglnatring Laboratory* tha 
Halttaad matrlct* McCaba't cyclomatir complaiity and varlout 
ttandard matrict hava baan analy^ad for thair ralation to 
afforti davalopaiant arrort and ona anothar CBat82al Thit 
ttudy aka^'.inad data collactad from tavan SEL (FORTRAN) pro> 
jactt and appliad thraa affort raporting accuracy chacht to 
damonatrata tha naad to validata a databata. 

Tha invaatigatlon axaminad tha corralatlont of tha 
''arioiit ffiatrlci with affort (functional tpacif icationt 
through accaptanca tatting) and Javalopmant arrort (both 
ditcrata and walghtad according to amoint of tima to locata 
and fix) acrott tavaral projact-i at onca. within individual 
projactt and for individual programma^'i acrott projactt 

In ordar to ramova tha dapandanct^ of tha dittribution 
of tha corralatlon coafficiantt on tha actual maaturat of 
affort and arrort* tha non-paramatr Ic Spaarman rank-ordar 
corralatlon coafficiantt wara tkaminad CKan791 Tha 
matrlct' corralationt with actual affort taam to ba itrong- 
att whan modulat davalopad antlraly by individual program^- 
mart or tahan froj .artaln valldatad crojactt ara con- 
tidarad Whan aiamining modulat davalopad totally by mdi- 
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vidu«l prografflmtrt# two avtrogta formtd from tho propottd 
validity ratios indue# a statistically significai.t ordtring 
of tha magnitud# of s#v#ral of th# matrics' corrolations. 

Th# lystamatic application of on# of th# data r«liability 
chocks (th# fr«qu#ncy of offort roporting) substantial ly 
improves #ith#r all or s#v#ral of th# projects' effort 
corralation with the metrics. In addition to these rela- 
tionshipsi the Halstead metri: s seem to possess reasonable 
correspondence with their estimators* although some of them 
have size dependent properties. In comparing the strongest 
correlations* neither Halstead's E metric* McCabes' 
cyclomatic complexity nor source lines of code relates con- 
vincingly better with effort than the others. 

1he metrics examined in this study were calculated from 
primitive measures derived from a source analyzing program 
(SAP — Revision I) CDec821 An earlier version of this 
static analyzer implemented a less comprehensive definition 
of Halstead operators and operandsCO 'Ne781. Some work has 
been done comparing the metrics' correlations when they have 
been determined from the different interpretations of the 
primitive measures. 

This investigation has been submitted for publication 
to the Transactions on Software Engineering and will appear 
as a University of Maryland Technical Report. 
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